{
  "cells": [
    {
      "cell_type": "raw",
      "id": "1957f5cb",
      "metadata": {
        "vscode": {
          "languageId": "raw"
        }
      },
      "source": [
        "---\n",
        "sidebar_label: Weaviate\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ef1f0986",
      "metadata": {},
      "source": [
        "# WeaviateStore\n",
        "\n",
        "[Weaviate](https://weaviate.io/) is an open source vector database that stores both objects and vectors, allowing for combining vector search with structured filtering. LangChain connects to Weaviate via the weaviate-client package, the official Typescript client for Weaviate.\n",
        "\n",
        "This guide provides a quick overview for getting started with Weaviate [vector stores](/docs/concepts/#vectorstores). For detailed documentation of all `WeaviateStore` features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain_weaviate.WeaviateStore.html)."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c824838d",
      "metadata": {},
      "source": [
        "## Overview\n",
        "\n",
        "### Integration details\n",
        "\n",
        "| Class | Package | [PY support](https://python.langchain.com/docs/integrations/vectorstores/weaviate/) |  Package latest |\n",
        "| :--- | :--- | :---: | :---: |\n",
        "| [`WeaviateStore`](https://api.js.langchain.com/classes/langchain_weaviate.WeaviateStore.html) | [`@langchain/weaviate`](https://npmjs.com/@langchain/weaviate) | ✅ |  ![NPM - Version](https://img.shields.io/npm/v/@langchain/weaviate?style=flat-square&label=%20&) |"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "36fdc060",
      "metadata": {},
      "source": [
        "## Setup\n",
        "\n",
        "To use Weaviate vector stores, you'll need to set up a Weaviate instance and install the `@langchain/weaviate` integration package. You should also install the `weaviate-client` package to initialize a client to connect to your instance with, and the `uuid` package if you want to assign indexed documents ids.\n",
        "\n",
        "This guide will also use [OpenAI embeddings](/docs/integrations/text_embedding/openai), which require you to install the `@langchain/openai` integration package. You can also use [other supported embeddings models](/docs/integrations/text_embedding) if you wish.\n",
        "\n",
        "```{=mdx}\n",
        "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
        "import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
        "\n",
        "<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
        "\n",
        "<Npm2Yarn>\n",
        "  @langchain/weaviate @langchain/core weaviate-client uuid @langchain/openai\n",
        "</Npm2Yarn>\n",
        "```\n",
        "\n",
        "You'll need to run Weaviate either locally or on a server. See [the Weaviate documentation](https://weaviate.io/developers/weaviate/installation) for more information.\n",
        "\n",
        "### Credentials\n",
        "\n",
        "Once you've set up your instance, set the following environment variables:\n",
        "\n",
        "```typescript\n",
        "// If running locally, include port e.g. \"localhost:8080\"\n",
        "process.env.WEAVIATE_URL = \"YOUR_WEAVIATE_URL\";\n",
        "// Optional, for cloud deployments\n",
        "process.env.WEAVIATE_API_KEY = \"YOUR_API_KEY\";\n",
        "```\n",
        "\n",
        "If you are using OpenAI embeddings for this guide, you'll need to set your OpenAI key as well:\n",
        "\n",
        "```typescript\n",
        "process.env.OPENAI_API_KEY = \"YOUR_API_KEY\";\n",
        "```\n",
        "\n",
        "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
        "\n",
        "```typescript\n",
        "// process.env.LANGSMITH_TRACING=\"true\"\n",
        "// process.env.LANGSMITH_API_KEY=\"your-api-key\"\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93df377e",
      "metadata": {},
      "source": [
        "## Instantiation"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "27252956",
      "metadata": {},
      "source": [
        "### Connect a weaviate client\n",
        "\n",
        "In most cases, you should use one of the connection helper functions to connect to your Weaviate instance:\n",
        "\n",
        "- connectToWeaviateCloud\n",
        "- connectToLocal\n",
        "- connectToCustom"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dc37144c-208d-4ab3-9f3a-0407a69fe052",
      "metadata": {
        "tags": []
      },
      "outputs": [],
      "source": [
        "import { WeaviateStore } from \"@langchain/weaviate\";\n",
        "import { OpenAIEmbeddings } from \"@langchain/openai\";\n",
        "import weaviate from \"weaviate-client\";\n",
        "\n",
        "const embeddings = new OpenAIEmbeddings({\n",
        "  model: \"text-embedding-3-small\",\n",
        "});\n",
        "\n",
        "const weaviateClient = weaviate.connectToWeaviateCloud({\n",
        "   clusterURL: process.env.WEAVIATE_URL!, \n",
        "\t options : {\n",
        "      authCredentials: new weaviate.ApiKey(process.env.WEAVIATE_API_KEY || \"\"),\n",
        "      headers: {\n",
        "        \"X-OpenAI-Api-Key\": process.env.OPENAI_API_KEY || \"\",\n",
        "        \"X-Cohere-Api-Key\": process.env.COHERE_API_KEY || \"\",\n",
        "      },\n",
        "    },\n",
        "});"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "068c0312",
      "metadata": {},
      "source": [
        "### Initiate the vectorStore\n",
        "To create a collection, specify at least the collection name. If you don't specify any properties, `auto-schema` creates them."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "96250f2c",
      "metadata": {},
      "outputs": [],
      "source": [
        "const vectorStore = new WeaviateStore(embeddings, {\n",
        "  client: weaviateClient, \n",
        "  // Must start with a capital letter\n",
        "  indexName: \"Langchainjs_test\",\n",
        "});"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "59d64eb9",
      "metadata": {},
      "source": [
        "To use Weaviate's named vectors, vectorizers, reranker, generative-models etc., use the `schema` property when enabling the vector store. The collection name and other properties in `schema` will take precedence when creating the vector store."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6f2b4c50",
      "metadata": {},
      "outputs": [],
      "source": [
        "const vectorStore = new WeaviateStore(embeddings, {\n",
        "  client: weaviateClient, \n",
        "  schema: {\n",
        "    name: \"Langchainjs_test\",\n",
        "    description: \"A simple dataset\",\n",
        "    properties: [\n",
        "      {\n",
        "        name: \"title\",\n",
        "        dataType: dataType.TEXT,\n",
        "      },\n",
        "      {\n",
        "        name: \"foo\",\n",
        "        dataType: dataType.TEXT,\n",
        "      },\n",
        "    ],\n",
        "    vectorizers: [\n",
        "      vectorizer.text2VecOpenAI({\n",
        "        name: \"title\",\n",
        "        sourceProperties: [\"title\"], // (Optional) Set the source property(ies)\n",
        "        // vectorIndexConfig: configure.vectorIndex.hnsw()   // (Optional) Set the vector index configuration\n",
        "      }),\n",
        "    ],\n",
        "    generative: weaviate.configure.generative.openAI(),\n",
        "    reranker: weaviate.configure.reranker.cohere(),\n",
        "  },\n",
        "});"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ac6071d4",
      "metadata": {},
      "source": [
        "## Manage vector store\n",
        "\n",
        "### Add items to vector store\n",
        "\n",
        "**Note:** If you want to associate ids with your indexed documents, they must be UUIDs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "17f5efc0",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[\n",
            "  '610f9b92-9bee-473f-a4db-8f2ca6e3442d',\n",
            "  '995160fa-441e-41a0-b476-cf3785518a0d',\n",
            "  '0cdbe6d4-0df8-4f99-9b67-184009fee9a2',\n",
            "  '18a8211c-0649-467b-a7c5-50ebb4b9ca9d'\n",
            "]\n"
          ]
        }
      ],
      "source": [
        "import type { Document } from \"@langchain/core/documents\";\n",
        "import { v4 as uuidv4 } from \"uuid\";\n",
        "\n",
        "const document1: Document = {\n",
        "  pageContent: \"The powerhouse of the cell is the mitochondria\",\n",
        "  metadata: { source: \"https://example.com\" }\n",
        "};\n",
        "\n",
        "const document2: Document = {\n",
        "  pageContent: \"Buildings are made out of brick\",\n",
        "  metadata: { source: \"https://example.com\" }\n",
        "};\n",
        "\n",
        "const document3: Document = {\n",
        "  pageContent: \"Mitochondria are made out of lipids\",\n",
        "  metadata: { source: \"https://example.com\" }\n",
        "};\n",
        "\n",
        "const document4: Document = {\n",
        "  pageContent: \"The 2024 Olympics are in Paris\",\n",
        "  metadata: { source: \"https://example.com\" }\n",
        "}\n",
        "\n",
        "const documents = [document1, document2, document3, document4];\n",
        "const uuids = [uuidv4(), uuidv4(), uuidv4(), uuidv4()];\n",
        "\n",
        "await vectorStore.addDocuments(documents, { ids: uuids });"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dcf1b905",
      "metadata": {},
      "source": [
        "### Delete items from vector store\n",
        "\n",
        "You can delete by id as by passing a `filter` param:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "ef61e188",
      "metadata": {},
      "outputs": [],
      "source": [
        "await vectorStore.delete({ ids: [uuids[3]] });"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c3620501",
      "metadata": {},
      "source": [
        "## Query vector store\n",
        "\n",
        "Once your vector store has been created and the relevant documents have been added you will most likely wish to query it during the running of your chain or agent. \n",
        "In weaviate's v3, the client interacts with `collections` as the primary way to work with objects in the database. The `collection` object can be re-used throughout the codebase\n",
        "### Query directly\n",
        "\n",
        "Performing a simple similarity search can be done as follows. The `Filter` helper class makes it easier to use filters with conditions. The v3 client streamlines how you use `Filter` so your code is cleaner and more concise. \n",
        "\n",
        "See [this page](https://weaviate.io/developers/weaviate/api/graphql/filters) for more on Weaviate filter syntax."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "aa0a16fa",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "* The powerhouse of the cell is the mitochondria [{\"source\":\"https://example.com\"}]\n",
            "* Mitochondria are made out of lipids [{\"source\":\"https://example.com\"}]\n"
          ]
        }
      ],
      "source": [
        "import { Filters } from \"weaviate-client\";\n",
        "\n",
        "const collection = client.collections.use('Langchainjs_test');\n",
        "\n",
        "const filter = Filters.and(collection.filter.byProperty(\"source\").equal(\"https://example.com\"))\n",
        "\n",
        "const similaritySearchResults = await vectorStore.similaritySearch(\"biology\", 2, filter);\n",
        "\n",
        "for (const doc of similaritySearchResults) {\n",
        "  console.log(`* ${doc.pageContent} [${JSON.stringify(doc.metadata, null)}]`);\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3ed9d733",
      "metadata": {},
      "source": [
        "If you want to execute a similarity search and receive the corresponding scores you can run:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "id": "5efd2eaa",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "* [SIM=0.835] The powerhouse of the cell is the mitochondria [{\"source\":\"https://example.com\"}]\n",
            "* [SIM=0.852] Mitochondria are made out of lipids [{\"source\":\"https://example.com\"}]\n"
          ]
        }
      ],
      "source": [
        "const similaritySearchWithScoreResults = await vectorStore.similaritySearchWithScore(\"biology\", 2, filter)\n",
        "\n",
        "for (const [doc, score] of similaritySearchWithScoreResults) {\n",
        "  console.log(`* [SIM=${score.toFixed(3)}] ${doc.pageContent} [${JSON.stringify(doc.metadata)}]`);\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dced353b",
      "metadata": {},
      "source": [
        "### Hybrid Search\n",
        "In Weaviate, `Hybrid search` combines the results of a vector search and a keyword (BM25F) search by fusing the two result sets. To change the relative weights of the keyword and vector components, set the `alpha` value in your query. \n",
        "\n",
        "Check __[docs](https://weaviate.io/developers/weaviate/search/hybrid)__ for the full list of hybrid search options. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1c10440e",
      "metadata": {},
      "outputs": [],
      "source": [
        "const results = await vectorStore.hybridSearch(\"biology\", \n",
        "  {\n",
        "    limit: 1,\n",
        "    alpha: 0.25,\n",
        "    targetVector: [\"title\"],\n",
        "    rerank: {\n",
        "      property: \"title\",\n",
        "      query: \"greeting\",\n",
        "    },\n",
        "});"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "221b9a65",
      "metadata": {},
      "source": [
        "### Retrieval Augmented Generation (RAG)\n",
        "Retrieval Augmented Generation (RAG) combines information retrieval with generative AI models.\n",
        "\n",
        "In Weaviate, a RAG query consists of two parts: a search query, and a prompt for the model. Weaviate first performs the search, then passes both the search results and your prompt to a generative AI model before returning the generated response.\n",
        "   * @param query The query to search for.\n",
        "   * @param options available options for performing the hybrid search\n",
        "   * @param generate available options for the generation. Check docs for complete list"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "14d08e92",
      "metadata": {},
      "outputs": [],
      "source": [
        "const results = await vectorStore.generate(\"hello world\",\n",
        "    {\n",
        "        singlePrompt: {\n",
        "            prompt: \"Translate this into German: {title}\",\n",
        "        },\n",
        "        config: generativeParameters.openAI({\n",
        "            model: \"gpt-3.5-turbo\",\n",
        "        }),\n",
        "    },\n",
        "    {\n",
        "        limit: 2,\n",
        "        targetVector: [\"title\"],\n",
        "    }\n",
        ");"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0c235cdc",
      "metadata": {},
      "source": [
        "### Query by turning into retriever\n",
        "\n",
        "You can also transform the vector store into a [retriever](/docs/concepts/retrievers) for easier usage in your chains. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "id": "f3460093",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[\n",
            "  Document {\n",
            "    pageContent: 'The powerhouse of the cell is the mitochondria',\n",
            "    metadata: { source: 'https://example.com' },\n",
            "    id: undefined\n",
            "  },\n",
            "  Document {\n",
            "    pageContent: 'Mitochondria are made out of lipids',\n",
            "    metadata: { source: 'https://example.com' },\n",
            "    id: undefined\n",
            "  }\n",
            "]\n"
          ]
        }
      ],
      "source": [
        "const retriever = vectorStore.asRetriever({\n",
        "  // Optional filter\n",
        "  filter: filter,\n",
        "  k: 2,\n",
        "});\n",
        "await retriever.invoke(\"biology\");"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e2e0a211",
      "metadata": {},
      "source": [
        "### Usage for retrieval-augmented generation\n",
        "\n",
        "For guides on how to use this vector store for retrieval-augmented generation (RAG), see the following sections:\n",
        "\n",
        "- [Tutorials: working with external knowledge](/docs/tutorials/#working-with-external-knowledge).\n",
        "- [How-to: Question and answer with RAG](/docs/how_to/#qa-with-rag)\n",
        "- [Retrieval conceptual docs](/docs/concepts/retrieval)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8a27244f",
      "metadata": {},
      "source": [
        "## API reference\n",
        "\n",
        "For detailed documentation of all `WeaviateStore` features and configurations head to the [API reference](https://api.js.langchain.com/classes/langchain_weaviate.WeaviateStore.html)."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "TypeScript",
      "language": "typescript",
      "name": "tslab"
    },
    "language_info": {
      "codemirror_mode": {
        "mode": "typescript",
        "name": "javascript",
        "typescript": true
      },
      "file_extension": ".ts",
      "mimetype": "text/typescript",
      "name": "typescript",
      "version": "3.7.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
